- 3+ years in a Big-data Engineer role;
- Sound knowledge of Hadoop Architecture and distributed data environments;
- Excellent functional knowledge and experience with big data components such as Kafka, Spark,Hadoop, Hbase, Hive;
- Experience in optimizing hadoop/hive settings for maximum throughput (includes compaction; hive metadata; TeZ Containerization;);
- Experience in shell scripting, have knowledge of scripting languages such as Python and be exposed to programming languages such as Java;
- Knowledge of Table definitions, file formats, UDF, Data Layout (Partitions and Buckets);
- Monitor Cluster performance, resource usage, backup and mirroring, provisioning and automation (Administration);
- ProdOps experience on Storm/Kafka.
- Experience in analyzing MapReduce jobs logs and error handling;
- AWS/Cloud: Sound knowledge of Cloud basics: S3, EMR. Need experience in identifying opportunities to improve processes, operations and automation using IaC;
- Experience ingesting data from multiple data sources such as REST API, SFTP flat files, Streaming data using Apache Storm, Kafka Streams or others etc.
- Degree in Computer Science, IT, or similar field; a Master's is a plus;
- Data engineering certification (e.g AWS Certified Big Data - Specialty) is a plus;
- Experience with Agile methodology
Company
Location
Lisbon - Portugal
Job type
Full-Time
Python Job Details
Meet our Team
We represent Hitachi Vantara to enterprise clients across industries, establishing business relationships to understand customer challenges so that we can deliver profitable business for Hitachi products, services and solutions. We collaborate as a team and cross-functionally to ensure the success of our customers; success that is celebrated and shared. Our solutions bring value to every line of business and we need people like you to build those deep relationships and to passionately articulate our value proposition.
What you bring to the team
Will be a plus:
Our Company
Hitachi Vantara is part of the Global Hitachi family. We balance innovation with an open, friendly culture and the backing of a long-established parent company, known for its ethical reputation. We guide customers from what's now to what's next by unlocking the value of their data and applications to solve their digital challenges, achieving outcomes that benefit both business and society.
Our people are our biggest asset, they drive our innovation advantage and we strive to offer a flexible and collaborative workplace where they can thrive. Diversity of thought is welcomed and our employee base is represented by several active Employee Resource Group communities. We offer industry leading benefits packages (flexible working, generous pension and private healthcare) and promote a creative and inclusive culture. If driving real change gives you a sense of pride and you are passionate about powering social good, we'd love to hear from you.
Our Values
We strive to create an inclusive environment for all and are open to considering home working, compressed/flexible hours and flexible arrangements. Get in touch with us to explore how we might be able to accommodate your specific needs.
We are proud to say we are an equal opportunity employer and welcome all applicants for employment without attention to race, colour, religion, sex, sexual orientation, gender identity, national origin, veteran or disability status. With Japanese roots going back over 100 years, our culture is founded on the values of our parent company expressed as the Hitachi Spirit:
Wa - Harmony, Trust, Respect
Makoto - Sincerity, Fairness, Honesty, Integrity
Kaitakusha-Seishin - Pioneering Spirit, Challenge
We represent Hitachi Vantara to enterprise clients across industries, establishing business relationships to understand customer challenges so that we can deliver profitable business for Hitachi products, services and solutions. We collaborate as a team and cross-functionally to ensure the success of our customers; success that is celebrated and shared. Our solutions bring value to every line of business and we need people like you to build those deep relationships and to passionately articulate our value proposition.
What you bring to the team
Will be a plus:
Our Company
Hitachi Vantara is part of the Global Hitachi family. We balance innovation with an open, friendly culture and the backing of a long-established parent company, known for its ethical reputation. We guide customers from what's now to what's next by unlocking the value of their data and applications to solve their digital challenges, achieving outcomes that benefit both business and society.
Our people are our biggest asset, they drive our innovation advantage and we strive to offer a flexible and collaborative workplace where they can thrive. Diversity of thought is welcomed and our employee base is represented by several active Employee Resource Group communities. We offer industry leading benefits packages (flexible working, generous pension and private healthcare) and promote a creative and inclusive culture. If driving real change gives you a sense of pride and you are passionate about powering social good, we'd love to hear from you.
Our Values
We strive to create an inclusive environment for all and are open to considering home working, compressed/flexible hours and flexible arrangements. Get in touch with us to explore how we might be able to accommodate your specific needs.
We are proud to say we are an equal opportunity employer and welcome all applicants for employment without attention to race, colour, religion, sex, sexual orientation, gender identity, national origin, veteran or disability status. With Japanese roots going back over 100 years, our culture is founded on the values of our parent company expressed as the Hitachi Spirit:
Wa - Harmony, Trust, Respect
Makoto - Sincerity, Fairness, Honesty, Integrity
Kaitakusha-Seishin - Pioneering Spirit, Challenge
More Developer Job Boards
Fullstack Developer Jobs Golang Jobs JavaScript Jobs Python Jobs React Jobs Rust Jobs Java Jobs